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Abstract 

In cases in which an original image is blind, a decoding method where both the image and the 
messages can be estimated simultaneously is desirable. We propose a spread spectrum watermark- 
ing model with image restoration based on Bayes estimation. We therefore need to assume some 
prior probabilities. The probability for estimating the messages is given by the uniform distri- 
bution, and the ones for the image are given by the infinite range model and 2D Ising model. 
Any attacks from unauthorized users can be represented by channel models. We can obtain the 
estimated messages and image by maximizing the posterior probability. 

We analyzed the performance of the proposed method by the replica method in the case of the 
infinite range model. We first calculated the theoretical values of the bit error rate from obtained 
saddle point equations and then verified them by computer simulations. For this purpose, we 
assumed that the image is binary and is generated from a given prior probability. We also assume 
that attacks can be represented by the Gaussian channel. The computer simulation retults agreed 
with the theoretical values. 

In the case of prior probability given by the 2D Ising model, we evaluated the decoding perfor- 
mance by computer simulations since the replica theory could not be applied. Results using the 2D 
Ising model showed that the proposed method with image restoration is as effective as the infinite 
range model for decoding messages. 

We compared the performances in a case in which the image was blind and one in which it was 
informed. The difference between these cases was small as long as the embedding and attack rates 
were small. This demonstrates that the proposed method with simultaneous estimation is effective 
as a watermarking decoder. 
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I. INTRODUCTION 



Digital watermarking is attracting attention for its potential application against the mis- 
use of digital content. The basic idea of digital watermarking is that some hidden messages 
or watermarks such as a copyright or user ID are invisibly embedded in digital cover content. 
For image watermarking, we need to pay attention to both the hidden messages and the 
images themselves. Either watermarks are simply embedded by adding them to the cover 



content 



2] , or the cover content is transformed by discrete cosine transform (DCT) [3J] or 
wavelet transform J4] and the watermarks are embedded in the transform domain. For the 
watermarks themselves, random binary bit or Gaussian sequences are usually used for the 
embedding jl-3]. The messages may be encoded jfi]. The spectrum spreading method is an 
efficient, robust method. In this paper, we consider a decoding algorithm for the spectrum 
spreading method. 

The basic spectrum spreading technique is also used in code division nultiple access 
(CDMA) [6], where multiple users can transmit their information at the same time and 
within the same cell. Multiuser interference needs to be considered for the CDMA multiuser 
demodulator pro blem. Recently Bayes optimum solutions have been proposed on statistical 
10 1 . In spread spectrum digital watermarking [l-3], watermarks are generated 



mechanics 



7- 



by spreading the messages. Stego images, which are marked images, are generated by 
embedding these watermarks in the original images. Attacks to or misuses of the stego 
images can be represented by channel models. We must estimate the hidden messages from 
tampered images while reducing multi-watermarks interference. 

In an informed case - that is, a case in which the original image is known to the decoder 
- we can determine the difference between the original and the tampered images. Using a 
framework of the Bayes estimation 7H9] , we can estimate these messages from the received 
messages by maximizing the posterior probability ll|. In contrast, in the blind case - 
that is, a case in which the original image is unknown - we need to estimate the original 
images from the tampered images. Watermarks are treated as noises against the image, and 
therefore, image estimation need to be applied to such a case. Assuming the prior probability 
of images, we introduce Bayes image estimation jol. [121 15 1 to the blind watermarking model. 
In order to estimate original images, we must assume the model used to generate the images. 
Natural images are usually represented as 8 bits per pixel. Using the least significant bit 



(LSB) or parity of the natural images, binary images can easily be generated. Embedding 
the watermarks into the binary images is now common [16]. In this paper, we use binary 
images. 

Performance of the blind digital watermarking model has not yet been sufficiently eval- 
uated. We therefore evaluate the average performance of this model. In particular, in the 
blind case, we propose a method in which both messages and the original image can be 
estimated at the same time. In order to evaluate the proposed method, we derive saddle 
point equations by the replica method and then calculate the theoretical bit error rate. For 
the theoretical evaluation using the replica method, we assume the infinite range model as 
prior probability of images. Moreover, we evaluate the case of the 2D Ising model as a prior 
probability by computer simulations. 



Now, we discuss the feasibility of representing original images by the infinite range model 



and 2D I sing model. Watermarking methods such as the wet paper code [16| and matrix em- 
bedding [17j methods assume that content consists of binary data. Specifically, the original 
images to be embedded are generated by calculating LSB or parity bits. Figure [T] shows the 
parity images generated from a natural image, where (a) is the original natural image and 
(b) shows the parity image from the uncompressed natural image of (a). The parity image 
in (c) is generated after JPEG compression of (a). The black and white pixels represent the 
parity bits and 1, respectively. From these images, we can find that the parity image (b), 
which is uniformly at random, can be seen as an image generated from the infinite range 
model, and the image (c), which has some clusters, can be seen as one from the 2D Ising 
model. Since we can evaluate our method in theory, it is reasonable to introduce some image 
generation models. 



The rest of this paper is organized as follows. Section [TT] gives an overview of our water- 
marking model. We explain that both messages and images can be estimated by maximizing 
the posterior probability. Section |TTT] describes the saddle point equations derived by the 
replica method in order to evaluate our method. Section IIVI shows the results obtained by 
theory and computer simulations. We conclude the paper in Section [V] 
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(a) Original image 



(b) Parity of uncompressed image (c) Parity of JPEG image 



FIG. 1. Sample of natural and parity images 

II. DIGITAL WATERMARKING MODEL 

We describe a basic watermarking model in an informed case and an image restoration 
model before proposing our blind watermarking model. 



A. Informed case 

When a decoder has been informed of an original image, the informed spread spectrum wa- 
termarking model can correspond to the CDMA model. K-bit messages s = (s l5 s 2 , • • • , sk) t 
are embedded in an original image in layers, where = ±1. We assume the prior probability 
of messages is a uniform distribution given by 

P ( s ) = ^- W 

Each message Sj is spread by a specific spreading code & = • • • , £,f) T , and watermarks 

are obtained by summing the K spread messages. The length of the spread codes - that 
is, the chip rate - is equal to the size of the image, N. Each element of spreading codes £f 
takes ±1 with probability 

P(tf = ±l) = ^. (2) 



Here, (£f ) 2 = 1. /x-th watermark is represented by 

«V = -7^E3V /i = l,2,---,.V (3) 



The stego image or marked image X is created by adding the watermark w to the original 
image /; that is, X M = + w^. We ignore any embedding errors, because they are almost 
always small enough to be negligent. 

Here, assume we have received a tampered stego image that is attacked by an illegal user. 
We can consider this attack the deterioration process of an image. Attacks can be represented 
as noise in the communication channel (5), [lfj 19 1 . We assume the channel is represented by 



the additive white Gaussian noise (AWGN) channel. Therefore, the conditional probability 
of the tampered image r given messages s is given by 



N 



P (r\s) = TT P (r M |s) oc exp 



1 



TV 

E 



o_2 V ft 
/<T ^=1 



w,, 



(4) 



where noise obeys the Gaussian distribution Af(0, <Tq). 

What we want to know is how many messages the decoder can retrieve from the tampered 
image. We therefore need to estimate messages s and then calculate the bit error rate. In 
order to estimate the messages, the posterior probability of messages s given the tampered 
image r should be computed. Since the true parameter a\ is unknown, we set a parameter 
as a 2 . From (CQ) and Bayes theorem, the posterior probability is given by 

P(r\s)P(s) 



P(s\r) 



E s P(r\s)P(s) 



N 

E 



2a 2 ^ v " 



Z = Tr exp 



1 



N 



2a' 



e - W ^Y 



(5) 
(6) 

(7) 



where Z is a normalization factor called a partition function. The watermark is a function 
of the messages s. Tr stands for the summation over s. 

s 

For a maximum a posteriori (MAP) estimation, the estimated messages s are given by 

s = arg max P (x\r) , (8) 

where x = (xi, X2, ■ ■ ■ , xk) t are variables that represent messages. For a maximum posterior 
marginal (MPM) estimation, the estimated messages s are given by 



Si = arg max ^ P (x\r) , 



(9) 



where summation J2x\xt is a summation over x excepting Xj. With that, we can obtain a 
Bayes optimum estimation. 



(a) ao = 1.0 



(b) «o = 1-5 



(c) a = 2.0 



FIG. 2. Images generated by infinite range model (256 x 256) 



B. Image restoration model 



It is difficult to formulate natural images. In the image restoration method based on 
Bayes estimation, the original images are assumed to be generated from some probability 



distribution 12L [LJ, uM- In this paper, we assume that the original images consist of N 

nnn 

pixels and that the pixels are binary [12|, [13|, H5J . Moreover, we consider the infinite range 
model and the 2D Ising model as image generating models. The prior probability of the 
infinite range model is given by 



P (/) oc exp 



JV ii<v 



(10) 



where parameter ao represents the smoothness of an image and the summation J2n<u runs 
over all pairs of different indexes /i, v. 

Figure |2] shows some images generated by the infinite range model with «o = 1.0,1.5, 
and 2.0. In the case of (a), the image looks like high-frequency snow noise, while with the 
larger a in (c), smooth images appear. 

For a while, leaving the watermarking scheme aside, we concentrate exclusively on image 
restoration from a tampered image. In fact, the embedding process of the watermarks can 
be considered a Gaussian channel. Therefore, we assume the deterioration process from the 
original image to the tampered image is a Gaussian channel. In this case, the probability of 
the tampered image r given the original image / is given by 



N 



P(r\f)=l[P(r^\f)(xexp 



N 



(ii) 



For the infinite range model, from Bayes theorem, the image maximizing the posterior 
probability, 



p (f\ r ) = ^ ex P 



1 



N 



2 + a 
N 



Z = Tr P(r\f)P(f) 



(12) 
(13) 



can be chosen as the estimation image. Since the true parameters o"Q,ao are unknown, 
parameters a 2 and a are used. 



C. Blind case 



When the original image is unknown or blind at the decoder, both the messages and the 
image should be estimated at the same time. This method requires the posterior probability 
of messages s and image / given the tampered image r. Since the probability of the 
tampered image r is given by 



P (r\s, f) oc exp 



N 



(14) 



and the prior probabilities are given by (JTj) and ffTOj) . the posterior probability can be given 
by 

P{r\s,f)P{s)P(f) 



P(s,f\r) 



E sJ P (r\s, f) P (s) P (/) 



z exp 



1 



N 



Z(T [i=l iV [1<U 



where 



Z = Tr P(r\s,f)P(f). 



(15) 
(16) 

(17) 



Constant P (s) is reducible. Since the true parameters a 2 and «o are unknown, parameters 
a 2 and a are used. Now, we rewrite the posterior probability in a different form using 
the Hamiltonian H(s,f) as P(s, f\r) = exp(— H(s, f)/a 2 )/Z. We can then obtain the 
Hamiltonian, 

K 



j K K 1 K N 

l P i=1 j= i VP i=i M =i 

1 N K aa 2 

H 7^ E E f^i S i ^7" E /m/"' 

V A M =l j=l ly p,<v 



(18) 



where f3 stands for embedding rate (5 = K/N, and 

i N I N 

^ = ^E^, ^ = ^EO> (i9) 



From MAP and MPM estimations, the estimated messages s and estimated image / are 
given by 

MAP : (s, f) = arg max P (x,g\r) , (20) 
MPM : Si = arg max ]T P (X\r) , (21) 

where x = (xi, x 2 , ■ ■ ■ , Xjc) t and g = (gi,g 2 , - ■ ■ , QnY stand for variables of messages and 
image, respectively. For MPM estimation, S = (si, • • • , s K) /i, • • • , /at) t stand for the true 
values of original messages and image, and X = (x T ,g T ) T represent the corresponding 
variables of messages and image. 



III. THEORETICAL EVALUATION 
A. Bit error rate 

The accuracy for estimated messages can be measured by bit error rate BER(m), as 

BER(m) = (22) 

where m represents the overlap between the original message Sj and the estimated message 
Si and is defined as 

K 

K 

Image quality is usually measured by peak signal-to- noise ratio (PSNR). However, since we 
deal with binary images, the image quality can also be measured by bit error rate BER(i?). 
The overlap, R, between the original image and the estimated image f ^ is defined as 



in=— Y, s i$i- ( 23 ) 
i=i 



1 N ~ 

iV *t=i 

Because mean squared error MSE = 4BER(_R), PSNR can be calculated from BER(i?). 

Using the bit error rate (BER), we evaluate the performance of our proposed method, 
which estimates both messages and image at the same time. We want to know the average 

9 



performance rather than specific messages and image. Therefore, we average the BER over 
all possible messages s, images /, and spread codes £f. We assume random diffusion by 
spread codes and a large system limit. Under this assumptions, we can derive saddle point 
equations of overlaps m and R from the posterior probability and can then theoretically 
evaluate the performance. 



B. Replica method 



In order to determine the average performance, Gibbs free energy F is averaged over 
messages, the pixel value of images, and spread codes. That is, [F] = —T[\ogZ], where [•] 
denotes a configurational average defined by 

[x] = [ n dr, Tr (P (r\a, f) P (s) P (/) x)^ , 

J u=l Sj 



(25) 



and (-)g denotes an average over £f. By using the replica method, we can obtain this 
averaged free energy [F] from the relation 

[Z n ] - 1 



[logZ] = lim 



(26) 



n->o n 

In other words, [log Z] can be calculated from n replicas of the original system using the 
configurational average of the product of the partition functions, Z n . We therefore start to 
calculate from 



r N l 

\Z n }= / \{drJTrP{r\sJ)P{s)P{f)Z n 
J ti=\ Xs,J " 

N If \-~ 



(27) 



N 

E 



K 



n N 



K 



2rr 2 L^i \ ' M 
2 



r„. - 



rrp E £l Si /j" 
V A i=l 



2o- 



EEfc 



fl<U 



,(28) 



where a is the replica index. 

n n 

According to the replica analysis of the CDMA model [7|, |9], we first need to carry out 
the terms of the messages. Let us average over the spread codes £f . By introducing the 
following notations to (125]) : 



K 



K 



E ^i^i ' 



(29) 
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we obtain 



[Z n }= [ dvZUdvZe N ^\ 



(30) 
(31) 



e N 92 = Tr n / 

f,9 ,, J 



dr. 



exp 



^2(^-<-/m) 2 -^E(^-<-^) 



+ AT E f U TVf E E 9fJ,9v 

ly fi<u ly a=l ii<v 



(32) 



In the term e Ngi , using the integral representation of delta function £(■), we can carry out 
the average over the spread codes £f and then introduce order parameters to the terms of 
the messages Sj,xf, given by 



1 K 1 K 

\ a b \ " a 

Qab — JT /^i X i X i ' 171(1 = ~j7 L^i SiX i " 
A i=l A i=l 



(33) 



The term e Ngi can be represented as 

eNai = I n / <w f - e W) n / - e 

S,!B [a<*> V i=l / a J \ i=l j 
TT f -,-r ^ 



E(^) 



E^«-E 

a<b a 



(34) 



x 



idq ab dq ab ^ idm a dm a 

a<b i 



-K Vabqab ~ i^E 



a<b 



1 



•« + i E *W - o E(^) 2 - E 



E™»*W-^) 2 



if 

x Tr TT exp 

s,a; x -*- 
k=l 



E lab^k^k + E m a s k%k 
a<b a 



(35) 



From integrating into the terms of r^fg, and Uq, we can obtain the term (1A5I) . 
(See appendix |A] for more details on this derivation.) Now, we assume symmetry between 
replicas for the order parameters of images; that is, q ab = q, q ab = q, m a = m, and m a = m. 
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Under this assumption, we obtain 



[z n ] 



idqdq idmdfn 
~27T 2tt ' 



,Af(Gi+G 2 +G 3 ) 



G\ = — n(n — l)Pqq — n/3mm, 



G 2 



n/- 



+ n/3 y L> 2 log 2 cosh 



+ m 



(36) 
(37) 
(38) 



Tr II 



dv^dv^ -r-j dv£dv% dr h 



2tt 5 2 7T v^a 



exp 



X + *EW-,ER) 2 - 



_ _^^ 2 



i 



-5 E ^ - m E VoV a - --2 (?> - v$ - f,,) 2 ~7T^J2 



a<b 



n 



where D 2 = dz/y/2ne~ z2 ^ 2 . By integrating over v£,v%, the term e^ 03 is given by 



(39) 



x exp 



* + *E^ + ^E<+^(E/,] +^E(E< 



V 



a 



N 



a<b 



2N 



2iV 



a \/i=l 



(40) 



(See appendix[B]for more details on this derivation.) This term represents contribution from 
the image. 

Next, for term e^ 3 , we introduce various order parameters of the images, given by 



N 



N 



r ° ~ E Ta ~ at E 9fj, 



N 



) fi i 



N 



AT 



Using these order parameters, we can rewrite it as 



.NG 3 



J dr J] dr a J] dR a J] dQ a6 e^ +G ^+^, 



(41) 
(42) 

(43) 



where 



Tr Tr exp 
/ 9 



a a a <b 



N 



N 



A" 



- E ^. ^ - E « - E a* ^ - E ^, 



a ,,6 
I- 1 



-,NG 5 



a n (a 2 + l-q 



n-\ N 



a<b 



(44) 
(45) 
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,NG 6 



exp N 



exp N 



n(2m- g -qg-l) + n(ag + l) +T|2EQab + n 



2(a 2 + 1 - g) 



2<X 2 



a<6 a Z Z a 



a<6 



(46) 



(47) 



The variables T, $, and f2 are given by (1B5|) - (1B8I) . We assume the replica symmetry for 
these order parameters; that is, r a = r, r a = r, R a = R, R a — R, Q a b = Q, and Q a b = Q- 
They lead to 



NG 3 f idr dr f idrdr f i d lid II j idQdQ s 



2tt 



2tt 



2tt 



(48) 



where 



G 4 = -r f - nrr - nRR QQ - -Q 

+ log |2 cosh (r ) + n Tr exp (r f) J D s log 2 cosh ( s\ '(} ■ r -\ Rf 



Gk = n log a — — 



n (Tq + 1 n 



2 a 2 



— — log I <t + 1 — q 



(49) 
(50) 



n(2m — q — a 2 — 1) n.(<7g + 1) 



G 7 = $ + 



2(<x 2 + 1 - g) 

2m — g — <x 2 — 1 



2a 2 
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„2 



2(a 2 + l-q) 2 a 2 (a 2 + l-q) 

n{n — 1) «o 2 na 

2 



+ 



»g + l 
2a 4 



-*Q + nQR + ^r 2 + ^r 2 



[1-Q + nQ), (51) 
(52) 



Thus, we obtain 



idqdq idmdm idr^dr^ idrdr idRdR idQdQ 



[ N{G 1 +G 2 +G 4 +G 5 +G( i +G 7 ) 



(53) 



2tt 2tt 2tt 2tt 2tt 2tt 
In the large-system limit N — > oo, the integral can be evaluated by the saddle point method. 
From (l2~6l . the free energy F is given in the limit n — > as 



F = -(3qq — (3mm — + (3 J D z log 2 cosh 



z\ q + m 



1 



2m — g — a 2 — 1 a 
2(a 2 + l-g) + T 



+ + ^r 2 - rf - i?£ - - (1 - Q) Q - (1 - Q) 



logo" - - log (a 2 + 1 - q 

2m — q — (Tq — 1 
2(a 2 + 1 - g) 2 



x _ R Tr exp (r /) / D s log 2 cosh [s yQ + r + Rf 



a 2 + 1 — q 2 cosh (fb) 

Since there are n-independent constant terms in F, we define them as 

-Fo = -r f + —rl + log 2 cosh(fo). 



(54) 



(55) 
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Extremization of the free energy yields the saddle point equations as 



m 
m 

Q 



R 



Q 
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D z tanh 
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zJq + m 



l-Q 
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1 



Tr e ?of D z tanh [ ar + z\J Q + Rf 



2cosh(f ) / 
1 



Tr fe rof D z tanh [ar + zJQ + Rf 



2cosh(f ) / 
1 



Tr e rof D z tanh 2 ar + z\jQ + Rf 



Q 



q — 2m + + 1 



a 2 + l-q ^ (a 2 + l-g) 2 
r = tanh(fo), f = a r . 



(56) 
(57) 
(58) 
(59) 

(60) 

(61) 

(62) 
(63) 
(64) 



In these equations, we can find two sets of equations for both the CDMA model [7H9| and 



the image restoration model [9(. These two equations depend on each other. 



0Jg| 



IV. SIMULATIONS 

A. Verification of saddle point equations 

We verify the obtained saddle point equations by computer simulations. First, we consider 
the infinite range model for the image restoration model. Figure |2] shows the original images 
generated that satisfy the prior probability (JTUJ), where ao = 1.0, 1.5, and 2.0. In the case of 
(b) a = 1.5, the average value of the pixels is r = 0.859 from (1641) . The message lengths 
are K = 32, 128, and 256, and the length of the spread codes is N = 256. Since image size is 
256 x 256, we make 256 blocks. Each block has N = 256 pixels, and then same watermark 
is embedded into the blocks. Figure [3] shows the bit error rate (BER) as a function of the 
channel noise. The parameters in the decoder, a, a 2 , are given by true values a = a and 
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q. The abscissa axis represents Eb/N Q given by 




(sO |dB| ' 



(65) 



where a 2 is the variance of the Gaussian channel. The axis of ordinate represents the BERs 
for both messages BER(m) and images BER(i£). BER(m) is calculated in each block and 
is averaged over all blocks. The average BER, BER(m), is shown with error bars. BER(i?) 
is calculated on whole image and is shown with points. The initial values of the estimated 
messages and estimated image are set by the true values, and then we obtain one of the best 
solutions. The theoretical values obtained by the saddle point equations are plotted by a solid 
line for embedding rate f3 = K/N = 0.125 (K = 32), a dashed line for (3 = 0.5 (K = 128), 
and a double-dashed line for (3 = 1.0 (K = 256). The computer simulations results agreed 
with those derived theoretically. In Fig. [31 the BER(m) for the messages worsened according 
to the embedding rate /3, while the BER(i?) for the images were slightly influenced by /3 
under the fixed smooth parameter a. 

Next, we evaluate the bit error rate for the smooth parameters a = 1.0, 1.5, and 3.0 under 
the fixed embedding rate /3. Figure H] shows the BERs for the embedding rate = 0.5. 
Because of the fixed embedding rate, the BER(m) for the messages were slightly influenced 
by the parameter a, while the BER(i?) for the images became better according to a. In 
other words, smoother images can be easily restored. 

B. Advantages of image restoration 

The key concept underlying the proposed method is that it can estimate both the mes- 
sages and image at the same time in the decoder. Here, we compare the performance of 
the blind decoder with that of the informed decoder. Cases in which the original image is 
known or informed to the decoder correspond to the CDMA model, and only messages are 
estimated. 

Figure |5] shows the bit error rate BER(m) for messages in the blind and informed decoders. 
The embedding rates are (3 = 0.125, 0.5, and 1.0. The BERs in the blind decoder are larger 
than those in the informed decoder because images are also estimated. However, in cases 
in which the embedding rate /3 is small enough, or in which there is not much noise in 
the communication channel, there is not much difference between the blind and informed 
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FIG. 3. (Color online) Bit error rate BER(m) and BER(i?) for a = 1.5. 



decoders, i.e., blind decoder can successfully carry out image estimation. 



C. 2D Ising Model 



In addition to the infinite range model, we also consider the 2D Ising model for image 
restoration. In this model, there are some clusters in generated images because the pixels 
interact with their nearest neighbors. These cluster patterns can be seen in the parity of 
JPEG images. In this section, we treat the 2D Ising model as an image generating model; 
that is, the prior probability is given by 



P (/) oc exp 



(66) 
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where (/i, v) denotes pairs of nearest neighbor sites. Figure |6] shows the generated images for 
parameters «o = 0.2, 1.0, and 1.5 in the 2D Ising model. In this manner, once the generating 
models have been changed, the generated images are much different. Since it is difficult to 
construct a generating model of natural images, it is necessary to consider various generating 
models in which as many characteristics of natural images are applied as possible. 

The posterior probability of the original image / given the tampered image r is given by 



P(f\r) 



1 



exp 



1 



2a 2 



N 

E 



(67) 



An exact solution of the 2D Ising model has not been obtained by the replica method yet. 
We therefore evaluate its performance by computer simulations. Figure [7] shows the bit 
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(a) a = 0.2 (b) a = 1.5 (c) a = 10.0 

FIG. 6. Images generated by 2D-Ising model (256 x 256) 



error rates BER(m) and BER(i?) for the 2D Ising model. The parameters a and a 1 are set 
to the true value a = cto and o 2 = a\. BER(i?) for images are slightly influenced by the 
embedding rate (3 under the fixed parameter a. 

Next, we evaluate the performance under the fixed embedding rate /3 = 0.25. Figure [8] 
shows BERs for the smooth parameters cto = 0.2, 1.5, and 10.0. BER(m) for messages are 
slightly influenced by a. BEK(R) for images in cases of «o > 1 are smaller than those of 
a = 0.4. For large «o = 10, phase transition may occure. 

Figure [9] shows the bit error rate BER(m) for messages in both blind and informed 
decoders. The embedding rates were (3 = 0.125,0.5, and 1.0. Curved lines denote the theo- 
retical values for the informed decoder. The blind decoder had just as good a performance 
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FIG. 7. (Color online) Bit error rates BER(m) and BER(i?) for 2D Ising model. 

as the informed decoder. That is, the blind decoder could successfully restore the image and 
estimate the messages. 



V. CONCLUSION 

We proposed an estimation method that can estimate messages and an image at the same 
time when using a blind decoder. When this method is used with Bayes estimation, prior 
probabilities for both the messages and the image are required. In this paper, we assumed 
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that the prior probability for messages had a uniform distribution and that those for images 
were the infinite range model and 2D Ising model. 

For the infinite range model, we derived the saddle point equations by the replica method 
in order to evaluate the average performance. Since there are two terms - the messages 
term and the image term - we implemented a two-step approach: first, we introduced 
order parameters for the messages and assumed replica symmetry for them, and second, we 
introduced order parameters for the image and assumed replica symmetry for them. The 
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obtained saddle point equations consist of two indivisible parts: the equations of the CDMA 
model and those of the image restoration model. We verified the saddle point equations by 
computer simulations. The theoretical results agreed with those of the simulations, except 
for the embedding rate /3 = 1.0. This difference, which may have been caused by replica 
symmetry breaking jlQl ]. will be the focus of our future investigation. 

Next, we evaluated the performance of the 2D Ising model by computer simulations. 
When the smooth parameter ao was fixed, there was little change in the bit error rate for 
images, and the bit error rate for messages depended on the embedding rate /3. In contrast, 
when the embedding rate f3 was fixed, there was little change in the bit error rate for 
messages, and the bit error rate for images depended on the smooth parameter. However, 
there was a lower bound in the 2D Ising model. 

We also evaluated the performance differences between blind and informed decoders. 
Results showed that the difference was very small when the embedding or attack rates were 
small, since the image restoration could still be carried out well. This demonstrates the 
effectiveness of the proposed method. 
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Appendix A: Integral of [Z n ] with respect to r^,VQ,v\ 



From (131)1) . we obtain 
where 
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Under the assumption of the replica symmetry, we obtain 
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Appendix B: Integral of with respect to v£,v% 



Under the assumption of the replica symmetry, we integrate by v%, and eliminate the 
terms at the limit n — > 0. We obtain 
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